SGI Varsity Update 1998 August

home *** CD-ROM | disk | FTP | other *** search

/ SGI Varsity Update 1998 August / SGI Varsity Update 1998 August.iso / dist / dist6.4 / mpi.idb / usr / share / catman / u_man / cat1 / MPI.z / MPI

Wrap

Text File | 1998-07-29 | 25KB | 524 lines

MPI(1) Last changed: 5-18-98 NNAAMMEE MMPPII - Introduction to the Message Passing Interface (MPI) IIMMPPLLEEMMEENNTTAATTIIOONN UNICOS, UNICOS/mk, and IRIX systems SSTTAANNDDAARRDDSS This release implements the MPI 1.2 standard, as documented by the MPI Forum in the spring 1997 release of _M_P_I: _A _M_e_s_s_a_g_e _P_a_s_s_i_n_g _I_n_t_e_r_f_a_c_e _S_t_a_n_d_a_r_d. The MPI implementation for UNICOS/mk systems is derived from the implementation of MPI for UNICOS MAX systems developed at Edinburgh Parallel Computing Centre. The software is supplied to Cray Research under license from the University of Edinburgh. DDEESSCCRRIIPPTTIIOONN The Message Passing Interface (MPI) is a component of the Cray Message Passing Toolkit and Message Passing Toolkit for IRIX (MPT), which is a software package that supports parallel programming across a network of computer systems through a technique known as message passing. The goal of MPI, simply stated, is to develop a widely used standard for writing message-passing programs. As such, the interface establishes a practical, portable, efficient, and flexible standard for message passing. In designing MPI, the MPI Forum sought to make use of the most attractive features of a number of existing message passing systems, rather than selecting one of them and adopting it as the standard. Thus, MPI has been strongly influenced by work at the IBM T. J. Watson Research Center, Intel's NX/2, Express, nCUBE's Vertex, p4, and PARMACS. Other important contributions have come from Zipcode, Chimp, PVM, Chameleon, and PICL. For UNICOS and IRIX systems, this release of MPI requires the presence of an Array Services daemon (aarrrraayydd) on each host that is to run MPI processes. In a single-host environment, no system administration effort should be required beyond installing and activating aarrrraayydd. However, users wishing to run MPI applications across multiple hosts will need to ensure that those hosts are properly configured into an array. For more information about Array Services, see the aarrrraayydd(1M), aarrrraayydd..ccoonnff(4), and aarrrraayy__sseerrvviicceess(5) man pages. When running across multiple hosts, UNICOS and IRIX users must set up their ..rrhhoossttss files to enable remote logins. (This does not apply to UNICOS/mk users). Note that MPI does not use rrsshh, so it is not necessary that rrsshhdd be running on security-sensitive systems; the ..rrhhoossttss file was simply chosen to eliminate the need to learn yet another mechanism for enabling remote logins. MPI is a specification (like C or Fortran) and there are a number of implementations. The rest of this man page describes the use of the Cray Research and Silicon Graphics implementations of MPI for UNICOS, UNICOS/mk, and IRIX systems. Other sources of MPI information are as follows: * Man pages for MPI library functions * A copy of the MPI standard as PostScript or hypertext on the World Wide Web at the following URL: http://www.mpi-forum.org/ * Other MPI resources on the World Wide Web, such as the following: http://www.mcs.anl.gov/mpi/index.html http://www.erc.msstate.edu/mpi/index.html http://www.osc.edu/lam.html GGeettttiinngg SSttaarrtteedd Software included in the Cray Message Passing Toolkit (MPT) for UNICOS and UNICOS/mk systems was designed to be used with the Cray Programming Environment 3.0 release or later. The MMoodduulleess software package is used to support the installation of both the Programming Environment and MPT. To use the MPT software, load the mmpptt module in addition to loading the Programming Environment module, PPrrggEEnnvv. For information on using modules, see _I_n_s_t_a_l_l_i_n_g _P_r_o_g_r_a_m_m_i_n_g _E_n_v_i_r_o_n_m_e_n_t _P_r_o_d_u_c_t_s, publication SG-5191, or, if the Programming Environment has already been installed on your system, see the online ASCII file, //oopptt//ccttll//ddoocc//RREEAADDMMEE. After you have initialized modules and loaded the PPrrggEEnnvv module, enter the following command: module load mpt To unload the mmpptt module, enter the following command: module unload mpt As an option for IRIX systems, MPT software can be installed in an alternate location for use with the modules software package. If MPT software has been installed on your system for use with modules, you can access the software with the mmoodduullee command shown in the previous example. If MPT has not been installed for use with modules, the software resides in default locations on your system (//uussrr//iinncclluuddee, //uussrr//lliibb, /usr/array/PVM,, aanndd ssoo oonn)),, aass iinn pprreevviioouuss rreelleeaasseess.. FFoorr ffuurrtthheerr iinnffoorrmmaattiioonn,, sseeee IInnssttaalllliinngg MMPPTT ffoorr UUssee wwiitthh MMoodduulleess,, iinn tthhee MMoodduulleess rreellnnootteess.. UUssiinngg SShhaarreedd MMeemmoorryy MMPPII oonn UUNNIICCOOSS SSyysstteemmss If you are using the shared memory method of communication, you must ensure that the correct data allocation occurs. In a multitasking environment, all global or static data is shared among the tasks in the multitasking group. Message passing assumes that this data is private to each task. To preserve this "private to a task" behavior, use the --hh ttaasskkpprriivvaattee option for C programs compiled with the cccc command, or the --aa ttaasskkccoommmmoonn option for Fortran programs compiled with the ff9900 command. These options specify that all global or static data is private to a task. For more information on the use of --hh ttaasskkpprriivvaattee and --aa ttaasskkccoommmmoonn, see the mmpptt__iinnttrroo(1) man page. The following example shows how to build a C program that will use the shared memory mode of communication: cccc --hh ttaasskkpprriivvaattee $$LLIIBBCCMM ffrreedd..cc LLIIBBCCMM is an environment variable that is automatically set to the following value when the MPT module is loaded: -D_MULTIP_ -L$MPTDIR/lib/multi It points to the reentrant version of lliibbcc..aa in the default MPT software. For more information on building programs that use shared memory, see the mmpptt__iinnttrroo(1) man page. Programs built to use shared memory communication can also be run by using TCP for communication, but the shared memory and TCP communication modes cannot be mixed during the execution of a program. To run a program that was built to use the shared memory mode of communication, use the mmppiirruunn(1) command to specify the number of tasks, as follows: mpirun -nt 4 fred UUssiinngg TTCCPP SSoocckkeettss ffoorr CCoommmmuunniiccaattiioonn oonn UUNNIICCOOSS SSyysstteemmss The following example shows how to build a program that will use TCP for communication: cc -o fred fred.c The environment automatically includes the path names to MPI include files and libraries. The MPI libraries are also automatically included. A program built for TCP will not run correctly in shared memory mode, but you can use TCP to run a program built for shared memory. To run a program that uses TCP for communication, use the mmppiirruunn(1) command and specify the number of processes by using the --nnpp option, as follows: mpirun -np 4 fred For more information about running TCP programs between multiple hosts, see the mmppiirruunn(1) man page. UUssiinngg MMPPII oonn UUNNIICCOOSS//mmkk SSyysstteemmss The following examples show how to build a program on UNICOS/mk systems: cc -o fred fred.c or f90 -o fred -X4 fred.f If you have loaded the mmpptt module, the directory that contains the MPI include files is automatically searched, and the MPI library is automatically loaded. When running MPI programs on UNICOS/mk systems, use of the mmppiirruunn(1) command is optional. You can choose to execute your program as you would any other UNICOS/mk application or you can use the mmppiirruunn(1) command. The following example shows how to execute a UNICOS/mk MPI program that has been built as a malleable executable file: mpprun -n 4 fred The following example shows how to execute a UNICOS/mk MPI program with mmppiirruunn(1): mmppiirruunn --nnpp 44 ffrreedd DDaattaa SSttrreeaammss oonn UUNNIICCOOSS//mmkk SSyysstteemmss UNICOS/mk hardware data streams are enabled by default for applications that use the Cray MPT implementation of MPI. MPI internal remote load and store operations are ensured to be coherent with streams operations. With data streams enabled, changes in behavior occur for the MMPPII__BBUUFFFFEERR__MMAAXX and MMPPII__BBUUFFFFEERR__TTOOTTAALL environment variables, as well as the MMPPII__BBuuffffeerr__aattttaacchh(3) and MMPPII__BBuuffffeerr__ddeettaacchh(3) routines. For information on these changes, see the mmppiirruunn(1), MMPPII__BBuuffffeerr__aattttaacchh(3), or MMPPII__BBuuffffeerr__ddeettaacchh(3) man pages. For more information on hardware streaming used on UNICOS/mk systems, see the iinnttrroo__ssttrreeaammss(3) man page. UUssiinngg MMPPII oonn IIRRIIXX SSyysstteemmss On IRIX systems, compile and link your MPI program as shown in the following examples. To use the 64-bit MPI library, enter the following command: cc -64 compute.c -lmpi To use the 32-bit MPI library, enter the following command: cc -n32 compute.c -lmpi Applications compiled under the previous (3.0) release of MPI should not require recompilation to run under this new (3.1) release. However, it is not possible for executable files running under the 3.0 release to interoperate with others running under the 3.1 release. The C version of the MMPPII__IInniitt(3) routine ignores the arguments that are passed to it and does not modify them. SSttddiinn is enabled only for those MPI processes with rank 0 in the MMPPII__CCOOMMMM__WWOORRLLDD environment variable (which does not need to be located on the same host as mmppiirruunn or xxmmppii). SSttddoouutt and ssttddeerrrr results are enabled for all MPI processes in the job. Unlike earlier releases, this version of the MPI implementation is compatible with the sspprroocc system call and can therefore coexist with ddooaaccrroossss loops. The MPI library is still not thread-safe, however. Therefore, calls to MPI routines in a multithreaded application will require some form of mutual exclusion. The IRIX implementation of MPI requires that all MPI processes call MMPPII__FFiinnaalliizzee eventually. BBuuffffeerriinngg oonn UUNNIICCOOSS aanndd IIRRIIXX SSyysstteemmss On UNICOS and IRIX systems, for the purposes of buffering, MPI messages are grouped into two classes based on length: short (messages with lengths of 64 bytes or less) and long (messages with lengths greater than 64 bytes). MPI buffers messages of all sizes. EENNVVIIRROONNMMEENNTT VVAARRIIAABBLLEESS This section describes the variables that specify the environment under which your MPI programs will run. Environment variables have predefined values. You can change some variables to achieve particular performance objectives; others are required values for standard-compliant programs. MMPPII__AARRRRAAYY Sets an alternative array name to be used for communicating with Array Services when a job is being launched. Default: The default name set in the aarrrraayydd..ccoonnff file Operating system: UNICOS and IRIX systems MMPPII__BBUUFFFFEERR__MMAAXX Specifies maximum buffer size. Specifies a maximum message size, in bytes, that will be buffered for MPI standard, buffered, or ready send communication modes. Default: No limit Operating system: UNICOS/mk systems MMPPII__BBUUFFFFEERR__TTOOTTAALL Specifies total buffer memory. Specifies a limit to the amount of memory the MPI implementation can use to buffer messages for MPI standard, buffered, or ready send communication modes. Default: No limit Operating system: UNICOS/mk systems MMPPII__BBUUFFSS__PPEERR__HHOOSSTT Determines the number of shared message buffers (16 KB each) that MPI is to allocate for each host. These buffers are used to send long messages. Default: 16 pages (1 page = 16KB) Operating system: IRIX systems MMPPII__BBUUFFSS__PPEERR__PPRROOCC Determines the number of private message buffers (16 KB each) that MPI is to allocate for each process. These buffers are used to send long messages. Default: 16 pages (1 page = 16KB) Operating system: UNICOS and IRIX systems MMPPII__BBYYPPAASSSS__DDEEVVSS Sets the order for opening HIPPI adapters. The list of devices does not need to be space-delimited (0123 is valid). An array node usually has at least one HIPPI adapter, the interface to the HIPPI network. The HIPPI bypass is a lower software layer that interfaces directly to this adapter. The bypass sends MPI control and data messages that are 16 Kbytes or shorter. When you know that a system has multiple HIPPI adapters, you can use the MMPPII__BBYYPPAASSSS__DDEEVVSS variable to specify the adapter that a program opens first. This variable can be used to ensure that multiple MPI programs distribute their traffic across the available adapters. If you prefer not to use the HIPPI bypass, you can turn it off by setting the MMPPII__BBYYPPAASSSS__OOFFFF variable. When a HIPPI adapter reaches its maximum capacity of four MPI programs, it is not available to additional MPI programs. If all HIPPI adapters are busy, MPI sends internode messages by using TCP over the adapter instead of the bypass. Default: 0123 Operating system: IRIX systems MMPPII__BBYYPPAASSSS__OOFFFF Disables the HIPPI bypass, forcing all host-to-host MPI messages to be sent through TCP. Default: Not enabled Operating system: IRIX systems MMPPII__CCHHEECCKK__AARRGGSS Enables checking of MPI function arguments. Segmentation faults might occur if bad arguments are passed to MPI, so this is useful for debugging purposes. Using argument checking adds several microseconds to latency. Default: Not enabled Operating system: UNICOS and IRIX systems MMPPII__CCOOMMMM__MMAAXX Sets the maximum number of communicators that can be used in an MPI program. Use this variable to increase internal default limits. (Might be required by standard-compliant programs.) MPI generates an error message if this limit (or the default, if not set) is exceeded. Default: 256 Operating system: UNICOS and IRIX systems MMPPII__DDIIRR Sets the working directory on a host. When an mmppiirruunn(1) command is issued, the Array Services daemon on the local or distributed node responds by creating a user session and starting the required MPI processes. The user ID for the session is that of the user who invokes mmppiirruunn, so this user must be listed in the ..rrhhoossttss file on the responding nodes. By default, the working directory for the session is the user's $$HHOOMMEE directory on each node. You can direct all nodes to a different directory (an NFS directory that is available to all nodes, for example) by setting the MMPPII__DDIIRR variable to a different directory. Default: $$HHOOMMEE on the node. If using the --nnpp or --nntt option of mmppiirruunn(1), the default is the current directory. Operating system: UNICOS and IRIX systems MMPPII__DDSSMM__MMUUSSTTRRUUNN Specifies the CPUs on which processes are to run. You can set the MMPPII__DDSSMM__VVEERRBBOOSSEE variable to request that the mmppiirruunn(1) command print information about where processes are executing. Using this feature might provide some benefit to your job at the cost of overall system performance for other jobs. Default: Not enabled Operating system: IRIX systems MMPPII__DDSSMM__OOFFFF Turns off nonuniform memory access (NUMA) optimization in the MPI library. Using this feature might provide some benefit to your job at the cost of overall system performance for other jobs. Default: Not enabled Operating system: IRIX systems MMPPII__DDSSMM__PPPPMM Sets the number of MPI processes that can be run on each node of an IRIX system. Only values of 1 or 2 are currently allowed. Default: 2 Operating system: IRIX systems MMPPII__DDSSMM__VVEERRBBOOSSEE Instructs mmppiirruunn(1) to print information about process placement for jobs running on nonuniform memory access (NUMA) machines (unless MMPPII__DDSSMM__OOFFFF is also set). Output is sent to ssttddeerrrr. Default: Not enabled Operating system: IRIX systems MMPPII__GGRROOUUPP__MMAAXX Determines the maximum number of groups that can simultaneously exist for any single MPI process. Use this variable to increase internal default limits. (This variable might be required by standard-compliant programs.) MPI generates an error message if this limit (or the default, if not set) is exceeded. Default: 256 Operating system: UNICOS and IRIX systems MMPPII__MMSSGGSS__PPEERR__HHOOSSTT Sets the number of message headers to allocate for MPI messages on each MPI host. Space for messages that are destined for a process on a different host is allocated as shared memory on the host on which the sending processes are located. MPI locks these pages in memory. Use the MMPPII__MMSSGGSS__PPEERR__HHOOSSTT variable to allocate buffer space for interhost messages. Caution: If you set the memory pool for interhost packets to a large value, you can cause allocation of so much locked memory that total system performance is degraded. Default: 1024 messages Operating system: IRIX systems MMPPII__MMSSGGSS__PPEERR__PPRROOCC Sets the maximum number of buffers to be allocated from sending process space for outbound messages going to the same host. (This variable might be required by standard-compliant programs.) MPI allocates buffer space for local messages based on the message destination. Space for messages that are destined for local processes is allocated as additional process space for the sending process. Default: 1024 Operating system: UNICOS and IRIX systems MMPPII__RREEQQUUEESSTT__MMAAXX Determines the maximum number of nonblocking sends and receives that can simultaneously exist for any single MPI process. Use this variable to increase internal default limits. (This variable might be required by standard-compliant programs.) MPI generates an error message if this limit (or the default, if not set) is exceeded. Default: 1024 Operating system: UNICOS and IRIX systems MMPPII__SSMM__PPOOOOLL Specifies the shared memory queue. When MPI is started, it allocates a pool of shared memory for use in message passing. This pool represents space used to buffer message headers and small messages while the receiving PE is doing computations or I/O. The default of 1024 bytes is the number of bytes that can be pending. Default: 1024 bytes Operating system: UNICOS/mk systems MMPPII__SSMM__TTRRAANNSSFFEERR Specifies the number of slots in the shared memory queue that can be occupied by a send operation at the receiver. A slot consists of four UNICOS/mk words. By default, a single send operation can occupy 128 slots (or buffer 512 words) while the receiving PE is doing computations or I/O. Default: 128 queue slots Operating system: UNICOS/mk systems MMPPII__TTYYPPEE__DDEEPPTTHH Sets the maximum number of nesting levels for derived data types. (Might be required by standard-compliant programs.) The MMPPII__TTYYPPEE__DDEEPPTTHH variable limits the maximum depth of derived data types that an application can create. MPI generates an error message if this limit (or the default, if not set) is exceeded. Default: 8 levels Operating system: UNICOS and IRIX systems MMPPII__TTYYPPEE__MMAAXX Determines the maximum number of data types that can simultaneously exist for any single MPI process. Use this variable to increase internal default limits. (This variable might be required by standard-compliant programs.) MPI generates an error message if this limit (or the default, if not set) is exceeded. Default: 1024 Operating system: UNICOS and IRIX systems SSEEEE AALLSSOO mmppiirruunn(1), mmpptt__iinnttrroo(1) aarrrraayydd(1M) iinnttrroo__ssttrreeaammss(3), MMPPII__BBuuffffeerr__aattttaacchh(3), MMPPII__BBuuffffeerr__ddeettaacchh(3), MMPPII__IInniitt(3) aarrrraayydd..ccoonnff(4) aarrrraayy__sseerrvviicceess(5) For more information about using MPI on UNICOS, UNICOS/mk, and IRIX systems, see the _M_e_s_s_a_g_e _P_a_s_s_i_n_g _T_o_o_l_k_i_t: _M_P_I _P_r_o_g_r_a_m_m_e_r'_s _M_a_n_u_a_l, publication SR-2197. Man pages exist for every MPI subroutine and function, as well as for the mmppiirruunn(1) command. Additional online information is available at hhttttpp::////wwwwww..mmccss..aannll..ggoovv//mmppii, including a hypertext version of the standard, information on other libraries that use MPI, and pointers to other MPI resources. This man page is available only online.